Properly Learning Poisson Binomial Distributions in Almost Polynomial Time

We give an algorithm for properly learning Poisson binomial distributions. A Poisson binomial distribution (PBD) of order n ∈ Z+ is the discrete probability distribution of the sum of n mutually independent Bernoulli random variables. Given Õ(1/ǫ) samples from an unknown PBD P, our algorithm runs in time (1/ǫ) , and outputs a hypothesis PBD that is ǫ-close to P in total variation distance. The sample complexity of our algorithm is known to be nearly-optimal, up to logarithmic factors, as established in previous work [DDS12]. However, the previously best known running time for properly learning PBDs [DDS12, DKS15b] was (1/ǫ), and was essentially obtained by enumeration over an appropriate ǫ-cover. We remark that the running time of this cover-based approach cannot be improved, as any ǫ-cover for the space of PBDs has size (1/ǫ) [DKS15b]. As one of our main contributions, we provide a novel structural characterization of PBDs, showing that any PBD P is ǫ-close to another PBD Q with O(log(1/ǫ)) distinct parameters. More precisely, we prove that, for all ǫ > 0, there exists an explicit collection M of (1/ǫ) log(1/ǫ)) vectors of multiplicities, such that for any PBD P there exists a PBD Q with O(log(1/ǫ)) distinct parameters whose multiplicities are given by some element of M, such that Q is ǫ-close to P. Our proof combines tools from Fourier analysis and algebraic geometry. Our approach to the proper learning problem is as follows: Starting with an accurate nonproper hypothesis, we fit a PBD to this hypothesis. More specifically, we essentially start with the hypothesis computed by the computationally efficient non-proper learning algorithm in our recent work [DKS15b]. Our aforementioned structural characterization allows us to reduce the corresponding fitting problem to a collection of (1/ǫ) log(1/ǫ)) systems of low-degree polynomial inequalities. We show that each such system can be solved in time (1/ǫ) , which yields the overall running time of our algorithm. Supported by EPSRC grant EP/L021749/1 and a Marie Curie Career Integration grant. Some of this work was performed while visiting the University of Edinburgh. Supported by EPSRC grant EP/L021749/1.